Skip to content

chore(sync): carry safe upstream fixes after fork main - #91

Merged
mrkillbob merged 20 commits into
mainfrom
codex/upstream-sync-20260912-latest
Sep 13, 2026
Merged

mrkillbob merged 20 commits into
mainfrom
codex/upstream-sync-20260912-latest

Conversation

@mrkillbob

Copy link
Copy Markdown
Owner

Summary

Scoped upstream sync based on mrkillbob/hermes-agent main at c01c61bc85d49c97cff5a543c8b24032e26fe041 (PR89 already merged). This deliberately does not merge upstream main wholesale; it carries only validated fixes that address the fork's unresolved review backlog while preserving fork-specific behavior.

Upstream reference: NousResearch/hermes-agent main at de2d6a1b93508463c31434c1ae067e204af81238.

Cherry-picked upstream commits

  • 3d7f773bb42498d66ea4c0be1fe7d2e0d00278d7 — Kanban/platform toolset recovery
  • 9b9026e62ff0607042bf6b76ac13e5bb41ea7c14 — Kanban explicit opt-in/TUI contract follow-up
  • 848075c501af5b4d77f42928bceea286f74640f4 — opaque AgentMail redaction (adapted to the fork's current PR89 redaction rules)
  • ceaf622c6d348c570b141898bee5916ca115a8d5 — profile-scoped MCP connection ownership (adapted to retain the fork's OAuth/mTLS owner-scope guard)
  • 819988acb750836387fbb9d5d76203a9b3f530f4 — Kanban per-platform opt-in documentation

Local compatibility commits:

  • ebf3a8eaf6cb090a78dd87613195ed0b73287d13 — align the existing worker-toolset expectation with explicit Kanban opt-in
  • 5bdf0078f28d774ca0308945888b273269e9975c — avoid lock recursion during MCP profile adoption

Already present on the fork base and therefore not duplicated: 4ddbcbd35ef03325fbabb10f6477a188b43339cd, c1ff9390f6a3e5da3cdd6210c445cb6d0aaa9fbf, 8aa773af89e67ea001ba4c61d84f77df33a8e6b9, and 5ff6cb0edbdf42a53b83d0e56e5b52a28a34d23e.

Validation

Focused scripts/run_tests.sh checks pass for Kanban, AgentMail redaction, backup, oneshot resume, MCP connection ownership, gateway multiplex MCP discovery, tool configuration, and TUI gateway coverage. Relevant Python ruff and bytecode compilation pass. The required full scripts/run_tests.sh run is in progress; its observed failures so far are unrelated baseline environment/host-guard failures (missing optional acp/anthropic packages and existing live-process guard tests).

No JavaScript/TypeScript files changed, so no JS-specific check is applicable.

This PR is intentionally separate from active repair PRs and is not to be merged until those repairs land.

Residual candidates not included

Later upstream backup commits (535fd88c70, b97ae5c9b8, 947e027f61, 3b3f354933, and 64fe13a647) broaden backup/cache/capture filesystem policy beyond the requested retention fix and were not pulled without a separate review. No later oneshot-specific commit was found. Later MCP include/desktop/multiplexer changes were broader than the scoped ownership repair and remain candidates for a separate sync review.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-13T07:01:44.895282Z 447bc88 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Sep 13, 2026 •

Copy link
Copy Markdown

૮ >ﻌ< ა ci review

ran on 447bc88 — fix(gateway): address remaining multiplex review feedback

⚠️ Warnings

OSV vulnerability scan · View job

8 known vulnerabilities found in pinned dependencies.

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.


debug info

CI timings

CI timings · View report · View job

Wall time 10m58s vs 10m54s (+0.6%). 9 job(s) slower, 16 faster, 1 unchanged.

  • Python tests / Run tests slice 6/8: +146.0s
  • Python tests / Run tests slice 8/8: +78.0s
  • Python tests / Run tests slice 7/8: -73.0s
  • Python tests / Run tests slice 3/8: +65.0s
  • Python tests / Run tests slice 1/8: -64.0s

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5bdf0078f2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/mcp_tool.py
Comment thread tools/mcp_tool.py
Comment thread website/docs/user-guide/multi-profile-gateways.md
@mrkillbob
mrkillbob force-pushed the codex/upstream-sync-20260912-latest branch from 5bdf007 to 58d067c Compare September 13, 2026 04:11

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 58d067c532

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/mcp_tool_registration.py Outdated
Comment thread tools/mcp_tool_discovery.py
Comment thread tools/mcp_tool_config.py
Comment thread tools/mcp_tool_scope.py Outdated
Comment thread hermes_cli/tools_config.py
@mrkillbob
mrkillbob force-pushed the codex/upstream-sync-20260912-latest branch 3 times, most recently from 2a93817 to 204258a Compare September 13, 2026 04:57

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 204258a37d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hermes_cli/gateway_migrate.py Outdated
Comment thread hermes_cli/gateway_migrate.py Outdated
Comment thread hermes_cli/gateway_migrate.py
Comment thread hermes_cli/gateway_multiplex_served.py
Comment thread hermes_cli/gateway_migrate.py
Comment thread hermes_cli/gateway_migrate.py
Comment thread hermes_cli/profile_channels.py
Comment thread hermes_cli/profiles.py
Comment thread hermes_cli/gateway_migrate.py Outdated
Comment thread hermes_cli/gateway_multiplex_served.py Outdated
Xipong and others added 11 commits September 12, 2026 22:25
…he opt-in contract

Salvage follow-up to Xipong's NousResearch#107736. Kept the core: `kanban` is a
configurable, default-off toolset whose check_fn answers the schema
build's own selection (ContextVar) instead of the legacy top-level
`toolsets` key, so `platform_toolsets.<platform>: [.., kanban]` — what
`hermes tools enable kanban --platform X` writes — actually reaches the
gateway agent's tool schema.

Dropped the `tui_gateway/server.py` change: turning an explicitly empty
CLI selection from "all" into "nothing" is a separate behaviour flip
already tracked by NousResearch#107452, not part of this bug. The two TUI loader
tests that asserted `kanban` is auto-recovered onto a saved `[memory]`
list now assert the opposite: a configurable opt-in is never recovered.

(cherry picked from commit 9b9026e)
…nly hex

Review finding: the salvaged rule assumed a lowercase-hex grammar that
AgentMail's docs do not establish (only the `am_` / `am_org_` prefix is
documented), so a non-hex key would have gone unmasked. Discriminate on
what actually separates keys from identifiers: an alphanumeric body with
no `_`/`-` and a 20-char floor. `am_example_identifier_123` still passes.

(cherry picked from commit 848075c)
…er profile; owner /reload-mcp keeps adopters' tools

Under gateway.multiplex_profiles every connection ledger in tools/mcp_tool.py
(_servers, _server_scope_keys/_server_tool_scopes, connecting/error/cooldown
maps, the circuit breaker, lazy schema-cache configs, trust metadata) was keyed
by the bare server NAME. The common per-tenant layout — each profile names its
server `github`/`notion` with its own token — gave only the first profile a
connection: the second profile's register_mcp_servers saw the name as "already
connected", refused to adopt it (different credentials, 4ddbcbd), and left
the profile silently tool-less with a healthy-looking `configured` status
(NousResearch#106005 Bug 1/2, NousResearch#91654). Siblings of the same bug: profile A's failing `x`
put profile B's healthy `x` into A's 10-minute connect cooldown and A's open
circuit breaker short-circuited B's calls; toolsets._resolve_toolset_memo was
not scope-keyed, so B resolved A's `mcp-<server>` tool names.

Keys are now the connection key from the new tools/mcp_tool_scope.py: the bare
name outside a multiplexer (single-profile processes are unchanged) and
(owner_scope, name) under one. Call-time lookups (_resolve_server_key) prefer
the calling scope's own connection, then a shared connection it adopted, so
identical-route profiles still share one subprocess. _select_new_servers,
the cooldown/breaker/trust maps, lazy registration and get_mcp_status all read
and write through the composite key; teardown resolves a task's key by
identity (the MCP loop has no profile context). The toolset memo key includes
registry.current_scope_key().

An owner's scoped /reload-mcp tore down its connection and, with it, every
adopting profile's tool overlay; nothing re-ran the adopters' discovery until
they reloaded. shutdown_mcp_servers(scope=) now records the orphaned adopters
and register_mcp_servers re-registers them under their own home + secret scope
once the owner's rediscovery pass completes.

Docs: multi-profile-gateways.md states the per-profile connection rule.

Fixes NousResearch#106005
Fixes NousResearch#91654
Co-authored-by: Bergmann89 <info@bergmann89.de>
Co-authored-by: Izzy-Gottz <srulynj@gmail.com>

(cherry picked from commit ceaf622)
…s enable kanban --platform X)

(cherry picked from commit 819988a)
…nels opts in

A cloned profile carried the source's TELEGRAM_BOT_TOKEN, DISCORD_BOT_TOKEN,
allowlists, WHATSAPP_ENABLED, API_SERVER_KEY and the platforms:/telegram:/
discord: config sections byte-for-byte. Standalone, that made two gateways
fight over one bot's long-poll; under multiplex it blocked
`hermes gateway migrate --multiplex` with one duplicate-credential finding
per platform per clone (18 on a real 10-profile install).

Every clone entry point (CLI --clone/--clone-from/--clone-all, dashboard
POST /api/profiles, TUI/Desktop profiles.create incl. its mirror_credentials
.env copy) now strips channel settings after the copy. The key set is derived
from the adapters — Platform enum + plugin registry (required_env,
allowed_users_env, allow_all_env, cron_deliver_env_var), the gateway env table
(gateway.config_env._ENV_STEPS / _ENV_ENABLE_CREDENTIALS) and each platform's
env prefix — so a new adapter is covered without a hand list. --clone-all also
drops pairing/WhatsApp-session/gateway ledgers. Provider and tool keys, the
model block, memory, skills and SOUL.md are untouched.

`--clone-channels` (REST/RPC: clone_channels) keeps them; it is refused when a
live multiplexer already serves the source and otherwise warns which
platforms are now shared. `hermes profile list` prints the same warning for
existing clones whose bot credential is byte-identical to the default's.

The dashboard's per-platform env-prefix table moves into profile_channels so
Channels-page cards and the clone stripper share one definition.
…-multiplex flips it with no standalone secondary

`hermes config set gateway.multiplex_profiles true` warned "not a recognized
config key" although gateway/config.py reads it: the key (and profile_routes)
were never in DEFAULT_CONFIG["gateway"]. Both are registered with their doc
comment; the CLI loaders deep-merge new keys, so no _config_version bump.

`hermes gateway migrate --multiplex` with two or more profiles but no
secondary running its own gateway printed "nothing to migrate" and left the
flag OFF. The explicit command now applies the one remaining step — flag on,
default gateway (re)started, the same rollback manifest (empty secondaries)
for --standalone. `hermes update`'s automatic hook keeps treating that case
as a no-op: it never flips modes on an install where nothing was running.
@mrkillbob
mrkillbob force-pushed the codex/upstream-sync-20260912-latest branch from 204258a to 25b3ac0 Compare September 13, 2026 05:35

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 25b3ac05bc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hermes_cli/gateway_migrate.py
Comment thread tools/mcp_tool_registration.py
Comment thread hermes_cli/profile_channels.py
Comment thread tools/mcp_tool_discovery.py
mrkillbob and others added 7 commits September 12, 2026 23:01
… unroutes deleted ones

A `gateway.multiplex_profiles` gateway enumerated `profiles/` once at boot, so a profile
created afterwards (CLI, dashboard, Desktop, TUI) was never served until `hermes gateway
restart`; Desktop and the dashboard gave no reminder, so a new profile's bot simply never
connected.

The served set is now reconciled at runtime (`gateway/run_profile_reconcile.py`):
- `hermes_cli/profiles.py` create/delete ping the multiplexer over its control socket
  (new `rescan-profiles` verb); a supervised watcher rescans every 30s as the safety net.
- A new profile gets its adapters under its own runtime scope from its config/.env
  (`_start_one_profile_adapters`, same duplicate-credential guard as boot, now seeded
  with the LIVE secondaries' claims), `served_profiles` in gateway_state.json is
  updated, MCP discovery + log routing run for it. Other profiles' adapters are never
  touched.
- A served profile whose config.yaml/.env changed is re-scanned so a token added after
  create builds the adapter; already-live/queued platforms are skipped (no second poller).
- A deleted profile (tombstone) has its reconnects cancelled, adapters torn down,
  pairing/busy bookkeeping and cached agents dropped, and this process's SQLite /
  memory-store handles released so the deleter's rmtree succeeds.
- The in-process cron ticker takes a live enumerator so new profiles' jobs fire.
- PUT /api/messaging/platforms/<id>?profile=X returns `hot_served` when a live
  multiplexer rebuilt X's adapters; Desktop/dashboard skip the restart banner then.
- `hermes profile create` confirms hot-serve; the restart reminder stays for a gateway
  that did not pick the profile up (older build / signal failed).

(cherry picked from commit d1dbb0a)
…iles leave no stale runtime entries

- PUT /api/messaging/platforms on a pooled `hermes --profile X serve` arrives without
  ?profile= (Desktop local topology, NousResearch#109088): resolve the hot-serve target from the
  process's own profile so the multiplexer is pinged and the UI skips the restart banner.
- A profile deleted while the reconcile lock was held by its own adapter connect was
  recorded back into served_profiles; re-check the live set before recording.
- Drop a deleted profile's `<name>:<platform>` runtime-status entries instead of leaving
  them as `stopped`.

(cherry picked from commit 2d121aa)
… start/install/status honour the live multiplexer

Under gateway.multiplex_profiles the default gateway serves every profile, yet
four startup/status paths still reasoned from the wrong source:

* A secondary profile's API_SERVER_KEY (which the docs REQUIRE for /p/<profile>/
  auth) auto-enabled api_server in that profile's config, so
  _load_secondary_profile_config raised SecondaryPortBindingConfigError and the
  whole profile was skipped. gateway/config_env.py::_enable_from_env now leaves
  `enabled` alone for port-binding platforms while a multiplexer loads a
  NON-default profile (home override + multiplex flag, the same signal
  gateway.config uses for scoped reads); the credential still lands in extra so
  the shared listener can authenticate the prefix. Default profile unchanged.

* "Is this profile served?" was re-derived from the default config.yaml plus
  GATEWAY_MULTIPLEX_PROFILES as seen by the CLI process. `hermes -p coder ...`
  loads coder's .env, so an env-only opt-in on the default profile was invisible
  (guard never fired, status said stopped) and an allowlist edit flipped the
  answer before the restart. named_profile_served_by_running_multiplexer now
  reads the pid-verified default gateway_state.json served_profiles (written by
  _record_served_profiles) first and falls back to config derivation only when
  the key is absent. The record helpers live in hermes_cli/gateway_multiplex_served.py.

* The served-profile guard ran only inside `gateway run`. `hermes -p X gateway
  start|install|restart` reached the service manager, whose unit then exited 78
  forever (systemd parks it while the CLI prints "started"; launchd KeepAlive
  respawns every 30 s). The service verbs now run the same guard up front
  (exit 78, same message) and accept --force; the Desktop /api/gateway/start
  route returns 409 for a served profile instead of spawning a doomed child.

* Status surfaces disagreed: `hermes -p coder status` said stopped, `hermes -p
  coder cron status` said "cron jobs will NOT fire" while `cron list` said fine,
  and the default `hermes status` never listed served profiles. Both now route
  through the probe / the recorded served set. The -p/--profile matcher in
  _scan_gateway_pids and gateway.status._command_line_belongs_to_profile compares
  the flag token for equality (`-p ops` no longer claims -- or lets `gateway
  stop` SIGTERM -- an `-p ops-2` gateway).

Docs: multi-profile-gateways.md now describes the start/install refusal, the
--force flags, the API_SERVER_KEY behaviour and the single default-home
gateway_state.json (the per-profile runtime_status.json claim was wrong).

Fixes NousResearch#100397
Addresses NousResearch#89726 NousResearch#97360 NousResearch#71344

(cherry picked from commit d002c1864a7b6a22c53758b16b7b0cc79aea2edf)
(cherry picked from commit 37dcc0a)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33b7e313e3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread website/docs/user-guide/multi-profile-gateways.md Outdated
Comment thread gateway/run_adapters.py
Comment thread gateway/run_profile_reconcile.py
Comment thread hermes_cli/gateway_migrate.py Outdated
Comment thread hermes_cli/gateway_multiplex_served.py Outdated
Comment thread hermes_cli/gateway_migrate.py Outdated
Comment thread gateway/run_profile_reconcile.py Outdated
Comment thread gateway/run.py Outdated
@mrkillbob
mrkillbob merged commit f41eb5e into main Sep 13, 2026
47 checks passed
@mrkillbob
mrkillbob deleted the codex/upstream-sync-20260912-latest branch September 13, 2026 06:57

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 447bc882ff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hermes_cli/gateway_migrate.py
Comment thread gateway/run_profile_reconcile.py
Comment thread hermes_cli/gateway_migrate.py
Comment thread hermes_cli/gateway_migrate.py
Comment thread hermes_cli/gateway_migrate.py
Comment thread hermes_cli/profile_channels.py
Comment thread gateway/run_profile_reconcile.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants